Field of the Invention
The present invention relates to a method and an apparatus for capturing an image of an object by using a camera. More particularly, the present invention relates to a technology for capturing an image of an object that moves with respect to a camera.
Description of the Related Art
In an automatic assembly line for a product using a robot, an operation in which the robot holds a component, which has been supplied, and mounts the component, which has been held by the robot, on a workpiece is repeated. In the case where the supplied component is not strictly positioned, measurement of the position and orientation of the component by using a camera may sometimes be performed in a process for holding the component.
This measurement is performed in order to bring the position and orientation of the component, which has been held by the robot, relative to the robot as close to a target position and a target orientation as possible. In other words, “a predetermined way in which the robot holds a component”, which is a target value, and “the actual way in which the robot holds a component”, which is an actually measured value, are compared with each other, and in the case where there is a difference between the target value and the actually measured value, the robot performs a process for causing the actually measured value to be the same as the target value. This process is performed in order to achieve an improvement in the accuracy with which assembly is performed by the robot in a process that is to be performed after the robot has held a component.
An exemplary specific method is as follows: the robot holds a component, then transports the component to a position in front of a camera, and after that, stops its operation. A still image of the component is captured by the camera, and the captured still image undergoes image processing, so that the position and orientation of the component relative to the robot are recognized. The robot performs a process for changing its orientation and a process for changing the way in which the component is held on the basis of the recognition results, so that the position and orientation of the component are changed to be within a range in which a process for mounting the component on a workpiece can be performed. After that, an assembly operation is performed.
In recent years, there has been a need to automatically assemble a product by using a robot with good efficiency while using visual information obtained by a camera. Thus, in order to achieve a further improvement in productivity per hour, it is effective to reduce as much as possible an increase in the time taken for transportation of a component due to slowing down, interruption, actuation, and speeding up of a robot, which occur during image capturing. Ultimately, it is desirable that an image of the component be captured while the component is moving in front of the camera without stopping the robot from transporting the component (this method will be hereinafter referred to as a method for capturing an image of a moving object).
In the method for capturing an image of a moving object, a sample image of an object (a target value) that has been previously stored and an image of the object captured while the object is moving (an actually measured value) are compared with each other, and the object to be measured is recognized in accordance with the level of similarity between the values.
In order to perform an image capturing of a moving object with high recognition accuracy, an image processing algorithm is determined, and in actual measurement, an image of an object needs to be captured while the object is moving at a resolution similar to that in a sample image of the object that has been stored as a target value in such a manner as to correspond to the image processing algorithm.
In addition, an image of the object needs to be captured under illumination conditions, such as reflections of shadows and ambient light, that are as similar to the illumination conditions under which the sample image has been acquired as possible.
For example, assume the case where an image processing algorithm that measures the position and orientation of a component, which is an object, by utilizing a mark placed on the component is used. In this case, in actual image capturing, the light of fluorescent lamps installed inside a factory may sometimes be reflected off a surface of the component, and glare may sometimes be produced. In addition, an image capturing may sometimes be performed under illumination conditions under which the mark is invisible to a camera due to the influence of such glare, and there is a possibility that the position and orientation of the component will be falsely recognized.
In the field of such image capturing of a moving object, as described in Japanese Patent Laid-Open No. 9-288060 (hereinafter referred to as Patent Document 1), a technology for extracting an image suitable for image analysis by capturing images of a moving component has been designed.
The following technology is described in Patent Document 1. In the technology described in Patent Document 1, a plurality of images of an object that is conveyed in front of a camera by a conveyor are continuously (or intermittently) captured in accordance with image-capturing-trigger signals sent from a control unit of the camera at regular intervals. After that, an image in which the distance between the object and the center of the image is the smallest is selected as the best image among the plurality of captured images, and then, the best image is output.
Operation of a camera described in Patent Document 1 is as follows: first, the last captured image and an image captured before the last captured image are stored in a memory of the camera. In this state, one of the following is selected on the basis of the displacement amount of the position of an object with respect to the center of the image, which is calculated in each of the two images: (1) the last captured image is determined as the best image, (2) the image captured before the last captured image is determined as the best image, and (3) the determination process is suspended, and the image capturing operation continues. In the case where the displacement amount falls below a predetermined threshold, that is, in the case where the object is sufficiently near the center of the image, (1) is selected. In the case where the displacement amount in the last captured image is larger than the displacement amount in the image captured before the last captured image, that is, in the case where the object, which has come near the center of the image once, has passed through the center of the image and is moving in a direction away from the center of the image, (2) is selected. In the case where neither of the above cases holds true, (3) is selected. As a result, in the case where capturing an image of a moving object is performed by using the technology of the related art described in Patent Document 1, an image in which the distance between the position of a component and a position predetermined by a user (hereinafter sometimes referred to as target capturing position) is the smallest can be retrieved from images captured by a camera.
However, in the method for capturing an image of a moving object described in Patent Document 1, an image to be output is selected among captured images while only the position of a component functions as a reference. Thus, in the case where an image of a component whose orientation has been inclined due to vibration and the like generated during transportation of the component is captured, the image of the component is captured under conditions under which the position and orientation relationship between the component and the camera/illumination is different from the position and orientation relationship when the image processing algorithm is determined. Therefore, an image captured under illumination conditions that are different from those originally assumed is acquired, and there is a possibility that the brightness in a portion of the acquired image may be different from the brightness that has been assumed, and that contour information of an area different from the area that has been assumed may be strongly detected. In such a case, in the case where a process for image processing and a process for recognition of an object are performed, there is a possibility that the position and orientation of the component will be falsely recognized, which in turn leads to failure of automatic assembly.